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ABSTRACT 


This paper presents a series of illustrative LEGO Mindstorms-based science and math activities, 
developed under an NSF GK-12 Fellows project, for elementary, middle, and high school grades. The 
activities, developed by engineering and science graduate Fellows in partnership with K-12 teachers, are 
grade appropriate, address pertinent learning objectives, and adhere to the science and math learning 
standards of New York City and State. To measure the effectiveness of the use of LEGO Mindstorms- 
based lab activities in science and math lessons, pre- and post-lesson assessment surveys, consisting 
of content and evaluation questions, were administered to all participating students. In this paper, we 
provide: our motivation to investigate the effectiveness of LEGO Mindstorms-based lessons; descriptions 
of six LEGO Mindstorms-based science and math activities and their associated assessment; statistical 
analysis; reflection on the effectiveness of the lessons; and recommendations for future work. 

Keywords: LEGO Mindstorms-based education, assessment, conceptual understanding 


INTRODUCTION 


As society continues its technological advancement at an exponential rate, maintaining competi¬ 
tiveness in the global economy requires that students at all levels develop technology proficiency 
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in proportion to the tempo of our changing world. In the United States, advances in technology 
have pervaded our daily lives and include the latest cellular phones that can generate step-by-step 
directions from one’s location to the nearest Starbucks and vehicles with voice activated systems 
that allow the driver to turn on the radio, adjust the mirrors, and control temperature, without ever 
lifting a finger. Although today’s students enjoy ready access to these technological advancements 
and effortlessly interact with such modern technological artifacts, they often lack a fundamental 
understanding of the underlying science and engineering concepts. Advancing students’ understand¬ 
ing requires pedagogical tools and techniques with the appeal of tech-savvy devices that capture 
students’ imagination while fostering their learning of science, technology, engineering, and math 
(STEM) principles. For over a decade, robotics competitions such as FIRST LEGO League (FLL) have 
provided a venue where students get an opportunity to explore and interact with advanced tools 
and devices used by engineers and technologists. In fact, on November 23, 2009, when President 
Obama introduced his new initiative, “To Educate and Innovate,” he said, “I believe that robotics can 
inspire young people to pursue science and engineering.” 

Examples of using robotics to teach STEM concepts abound in literature and cover the entire 
education spectrum from elementary to graduate school [1-4], Unfortunately, the extracurricular 
nature of robotics contests has not made the use of robotics more central to K-12 science and 
math education. Moreover, the potential for explicitly exploring science and math principles using 
robotics-based activities remains largely untapped in K-12 schools [5]. In fall 2010, we surveyed New 
York City (NYC) FLL coaches and received 43 responses (=33% response rate). The survey results 
revealed that approximately 50% of respondents do not use robotics in their classrooms and only a 
small number provided explicit, meaningful examples of their use of robotics in STEM classrooms. 
Many robotics-focused K-12 programs are organized as outreach efforts for students’ educational 
enrichment and necessitate on-site support of teachers through college-level engineering students 
[6] or volunteer engineering professionals [7], thereby making it difficult to sustain and scale-up 
projects. Therefore, increasingly, the focus of educators has been to transition students’ extra¬ 
curricular robotic experiences from the after-school and competition preparation programs to the 
classroom setting. 

This paper presents several illustrative LEGO Mindstorms-based science and math activities 
developed under a GK-12 Fellows project of NSF at the Polytechnic Institute of NYU (NYU-Poly). 
Over the past four years, 27 graduate Fellows pursuing masters or doctoral studies in engineering 
(e.g., chemical and biological, civil, electrical, computer, and mechanical), computer science, and 
chemical and biological sciences have participated in this project. The activities, developed by the 
NYU-Poly GK-12 Fellows in partnership with K-12 teachers, are grade appropriate, address pertinent 
learning objectives, and adhere to the science and math learning standards of New York City and 
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State. In the following sections, we provide: our motivation to investigate the effectiveness of LEGO 
Mindstorms-based science and math lessons; descriptions of six LEGO Mindstorms-based science 
and math activities; overview of assessment and evaluation methods used to analyze the effective¬ 
ness of the lessons; statistical analysis and observations; and reflection on the effectiveness of the 
lessons, discussion of sustainability, and recommendations for future work. 


MOTIVATION 


In his seminal work, [8], Papert showed that children can not only learn to use computers but 
also that learning to use computers helps shape the way they think, learn, and understand other 
disciplines, e.g., math. Papert’s work led to a series of investigations, such as [9-12], among oth¬ 
ers, on building children’s mathematical understanding through computers. Following Papert’s 
work, the use of modern technologies, e.g., robotics, in engaging the interest of K-12 students 
in STEM disciplines has been explored widely [1-7, 13-19]. Prior studies have yielded evidence 
that robotics-centered activities provide compelling opportunities for learning about the role of 
science and engineering in our society, including possible career opportunities in these fields, 
and life and workplace related skills, such as the ability to work with others on a team and time 
management [20]. Moreover, integration of sensor-based activities in science labs has led to 
increased student achievement in standardized tests, such as the New York State Living Environ¬ 
ment Regents examination [21], 

In this work, we have focused on integrating a variety of sensors, which are compatible with the 
LEGO Mindstorms NXT platform, to develop experimental apparatus that facilitate scientific explora¬ 
tions and broaden the use of student-friendly robotics technology [22-25], The LEGO Mindstorms 
platform offers a variety of components that not only help engage students’ creativity but also allow 
the application of teaching strategies such as scaffolding and problem-based learning. For example, 
judicious integration of sensors in hands-on lab activities can engage students’ understanding since 
it allows connecting abstract concepts or textbook formulae to tangible measurements performed 
by students. The variety of sensors available with the LEGO Mindstorms robotics platform permits 
the acquisition and processing of a multitude of physical stimuli arising in science subjects that often 
require separate, standalone equipment. Although LEGO Mindstorms kits represent an additional cost 
($300 per kit) for schools that do not participate in FLL competition, they are available in thousands 
of schools that do. In comparison to alternative technologies, such as pen lasers, photodetectors, 
and data loggers, LEGO Mindstorms are more user-friendly and less problem-prone during lesson 
preparation and deployment. 
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Although, mobile robotics can be used to illustrate and reinforce numerous science and math 
concepts, in practice, many K-12 robotics programs concentrate solely on mechanical design and 
programming aspects of the robot. Such an approach fails to exploit the motivational powers of 
robotics for students’ science and math learning. Specifically, in science and math classrooms, 
robotics technology can be presented to students as a tool to perform real-world experimental 
investigations, hence creating a bridge between the different STEM disciplines. For example, as 
students develop strategies for the locomotion of a robot traveling a specified distance, they 
gain an understanding of the geometry of a circle, which connects the distance traveled by the 
robot to the circumference of its wheels. Hence, such an approach requires a departure from the 
traditional tri-fold robotics curricula of construction, programming, and task-oriented applica¬ 
tion to one with a stronger emphasis on the factors that influence the behavior of the robot and 
the data collected by it. This alternative approach allows students to focus on analyzing trends 
or patterns in the data and making inferences or predictions when experimental parameters are 
changed. 

In this paper, we offer the overview and assessment of six science and math activities that em¬ 
ployed LEGO Mindstorms-based automated lab apparatuses and robotics as a tool to enhance 
understanding of relevant concepts of the lessons and, in some cases, aid in the reporting of ex¬ 
perimental data. These activities offer students authentic laboratory experiences that reinforce 
traditional classroom instruction, support deeper understanding of the subject matter [1, 2, 16, 17, 
25], and promote active earning through discovery [14], Additional theoretical and pedagogical 
justification for the approach of this paper is evidenced through Piaget’s constructivism [26-28], 
Papert’s constructionism [29], motivational aspect and problem-solving approaches supported 
by LEGO Mindstorms [5], versatility of LEGO Mindstorms as an experimental tool for elementary 
through graduate education [1-4], among others. 

The six LEGO Mindstorms-based lessons were developed and conducted in elementary, middle, 
and high school science and math classes in several public schools in Central Brooklyn. Specifically, 
the three science activities, viz.. The Mechanical Advantage, Acceleration due to Gravity, and Fluid 
Flow Rate, were conducted in elementary school science, middle school technology, and high school 
science integration classes, respectively. The three math activities, viz., Measurements & Accuracy, 
Pi-What is it? and Means, Modes, and Medians, were conducted in elementary school science and 
robotics, middle school technology, and high school algebra classes, respectively. The objective of 
the lessons was to teach students science and abstract math concepts, which are tested on New 
York State exams, through fun, engaging, and interactive hands-on LEGO Mindstorms-based activi¬ 
ties. The GK-12 Fellows and the teachers collaborated in conducting the LEGO Mindstorms-based 
experiments, which were used to reinforce the state mandated learning standards, during two con- 
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secutive 45 minute class sessions. The teachers reviewed and approved each lesson for alignment 
with their curriculum. 

Using student-friendly technology and software, the lessons promoted team-oriented and 
research-like environment. All lessons and corresponding evaluation instruments can be obtained 
by accessing NYU-Poly’s GK-12 project website. Moreover, video snippets of LEGO Mindstorms- 
based science and math lessons in action can be viewed here. This effort has three main goals: (1) 
to provide concrete illustrations of the versatility of LEGO Mindstorms in teaching K-12 level sci¬ 
ence and math concepts; (2) to exploit LEGO Mindstorms-based automated lab apparatuses and 
robotics to facilitate automated data collection for K-12 level experimental activities that otherwise 
use manual, tedious, and problem-prone data collection technologies; and (3) to demonstrate 
through assessment that robotics can be exploited to enhance the teaching of science and math 
in K-12 classrooms. 


DESCRIPTIONS OF SCIENCE ACTIVITIES 


The Mechanical Advantage 

Using the LEGO Mindstorms kit, an experimental apparatus, consisting of a wirelessly controlled 
mechanized elevator platform, is constructed to demonstrate the use of pulleys in producing me¬ 
chanical advantage (Figure 1). Students in science labs explore the setup by adding or removing 
pulleys from the mechanical system, adding weights to the platform, and controlling power input 
to the drive motors of the platform using a custom designed controller which uses the Bluetooth 
feature of the LEGO Mindstorms NXT brick. Students are asked to identify the pulleys in the setup 
and examine the role of pulleys by operating the setup with and without pulleys. The effects of use 
or absence of pulleys is qualitatively assessed and recorded by students as the system’s ability or 
inability to lift objects of a certain weight, the speed at which objects are lifted, and the tension along 
the strings holding the platform. This lab activity reinforces the concept of producing mechanical 
advantage through the use of pulleys and it promotes a more coherent and complete identification 
of pulleys as a simple machine. 

Acceleration due to Gravity 

Using the LEGO Mindstorms kit, an experimental apparatus is constructed that is used by stu¬ 
dents to measure the time to travel a specified distance by a free falling body (Figure 2). Students 
use a touch sensor, a rotational sensor, and two LEGO Mindstorms NXT bricks, to measure the 
time of flight for the falling object, at different release heights. A robotic gripper holds the object 


SUMMER 2012 


5 






ADVANCES IN ENGINEERING EDUCATION 

Enriching K-12 Science and Mathematics Education Using LEGOs 



Figure 7. (a) Schematic of the pulleys setup used for The Mechanical Advantage activity; 
(b) photo of setup; and (c) students performing the activity. 


and releases it upon receiving the operator command. When the object reaches the end-point of 
its travel, the touch sensor is triggered and the time of object’s descent from release to impact at 
the touch sensor is recorded and displayed on the LEGO Mindstorms NXT screen. Moreover, using 
several different release points, students calculate the corresponding average velocity of the falling 
object. Next, they plot a graph of average velocity versus time and apply a best fit line to this graph. 
Finally, students determine the slope of the best fit line (one-half g ) and compare it to the standard 
value of g (see Figure 2). This lab activity reinforces the concept of gravitational acceleration and 
an experimental method to determine this constant. 

Fluid Flow Rate 

Using a LEGO Mindstorms-based automated lab apparatus, this activity introduces students to 
the concepts of flow rate and its dependency on pipe diameter. A pair of LEGO Mindstorms light 
sensors is used in a photogate configuration. The designed setup functions in the manner of a 
stopwatch wherein the events are timed using the light sensor signals instead of manually operat¬ 
ing a stopwatch (Figure 3). A plastic bottle is used as a liquid reservoir and is fitted with a nozzle 
at its bottom to release the liquid. Upon release from the reservoir, the liquid falls into a glass bea¬ 
ker which acts as a basin. As the liquid draining from the reservoir fills up the basin to the level of 
light sensor #1, the “stopwatch” programmed on LEGO Mindstorms is triggered to initiate timing 
of the event. Next, the liquid level in the basin continues to rise and triggers light sensor #2 upon 
reaching its level. Triggering of light sensor #2 signals the termination of the stopwatch timer. The 
time elapsed between the triggering of two light sensors is recorded and displayed on the LEGO 
Mindstorms NXT screen and used to compute the average volumetric flow rate of the system. By 
attaching orifice fittings of various diameters to the bottom of the bottle, students discover the 
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Figure 2. (a) Experimental apparatus for the Acceleration due to Gravity activity; (b) 
students performing the activity; (c) schematic of motion for a free falling body; and (d) plot 
to determine g. 


effects of different diameter orifices on the average volumetric flow rate of the system. Similarly, by 
changing the initial level of liquid in the reservoir, students can examine the effect of initial liquid 
level on the average volumetric flow rate. 


DESCRIPTIONS OF MATH ACTIVITIES 


Measurements & Accuracy 

This activity introduces students to the concept of length measurement and data organization. 
In particular, students are taught to take accurate measurements, understand the importance of 
units, organize data in ascending and descending order, and compare collected data according to 
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the length. Using a LEGO Mindstorms robot outfitted with an attachment to accommodate a pen 
or marker, various line segments are drawn as specified by the number of rotations of the wheels 
of the robot (Figure 4). For example, when the robot is pre-programmed to move forward by one 
wheel rotation it draws a line of approximately 7 inches. Similarly, when the robot moves by two 
wheel rotations it draws a line of approximately 14 inches. Using a ruler, students are tasked with 
measuring the length of each line segment to the nearest half inch and recording their measure¬ 
ments using the appropriate units (Figure 4). 

Pi-What is it? 

This activity was developed to promote conceptual understanding of the irrational properties 
of 7t and its applications in geometry. The objective of the activity is to create an environment in 
which students assume the role of researchers seeking to establish an approximate value of n and 
understand its relationship to circumference and area of a circle. In addition, the students review 
the concept of ratios as well as rational and irrational numbers through hands-on applications. 
Using a LEGO Mindstorms robot outfitted with an attachment to accommodate a pen or marker, 
circles of various diameters are drawn as determined by a pre-loaded program (Figure 5). Students 
are tasked with measuring the radius or the diameter of the circle using a ruler and calculating the 
circumference and area of the circle along with several useful ratios (Figure 5), ultimately leading 
to the empirical exploration of 71 and its properties. 
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Figure 4. (a) Robotic setup used in the Measurements & Accuracy activity; (b) a student 
recording his measurements and filling out post-lesson evaluation survey; (c) measurements 
taken by a student; and (d) the table used to record and organize data during the activity. 


Means, Modes, and Medians 

Using a LEGO Mindstorms-based experimental apparatus, this activity introduces to students 
the concepts of empirical data collection and analysis. First, to produce periodic motion of an 
oscillating spring-mass system, a rubber band is suspended from a clamp-stand with a 1 kg mass 
and a cardboard platform rigidly attached to the inelastic part of the cord (Figure 6). Next, a LEGO 
Mindstorms ultrasonic sensor is placed on the base of the clamp-stand and above the platform to 
measure its distance from the oscillating cardboard platform. The data logging component of the 
LEGO Mindstorms software is used to record the changes in distance of the cardboard platform 
and output the values to a spreadsheet program for display and analysis. Students are tasked with 
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(a) (b) (c) 



Figure 5. (a) Robotic set up used in the Pi-What is it? activity; (b) students operating the 
robotic device; (c) a student taking measurements with the ruler and a measuring tape; (d) 
the drawn circle; and (e) the table used to record and calculate data during the activity. 


determining the mean, mode, and median for the collected data and evaluating the validity of the 
data points, such as the occurrence of outliers due to varying experimental conditions (Figure 6). 


EVALUATION METHODS 

To measure the effectiveness of the use of LEGO Mindstorms-based lab activities in science 
and math lessons, pre- and post-lesson assessment surveys (see Table 1), consisting of content 
and evaluation questions, were administered to all participating students immediately before 
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Figure 6. (a) Experimental apparatus used in the Means, Modes, and Medians activity; 

(b) students preparing to collect data; (c) students analyzing collected data from the Excel 
spread sheet; (d) data collected during the activity and presented on the Excel spread sheet, 
and (e) data recorded from the spread sheet. 


(pre-assessment) and immediately after (post-assessment) the lessons. The assessment surveys 
were developed by the GK-12 Fellows in collaboration with the teachers of respective grades and 
subjects to be timely, responsive, appropriate, and effective for the intended students. The GK-12 
Fellows also consulted with a science education expert on the appropriateness of survey questions 
and the development of activities to ensure that the pertinent learning standards for the subject 
and grade were adhered to [30, 31]. 

The survey utilized multiple-choice questions, fill in the blanks, and short answer essays in a student- 
friendly format. The evaluation questions sought to obtain information on students’ perception of 
science and math, their prior experiences with robotics, and their opinion about the LEGO Mindstorms- 
based science and math lessons (Table 2). The content questions were developed by drawing upon 
prior years’ New York City and State science and math assessment exams [32]. The content ques¬ 
tions sought to examine students’ knowledge gained through the LEGO Mindstorms-based activity 
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Activity 

The Mechanical 

Advantage 

Science, 

Elementary school 


Acceleration due 
to Gravity 
Science, Middle 
school 

Fluid Flow Rate 
Science, High 
school 


Measurements & 
Accuracy 
Science and 
robotics, 

Elementary school 


Pi-What is it? 
Math, Middle 
school 


Means, Modes, 
and Medians 
Math, High school 


Content Questions 

• What is a pulley and what is the purpose of 
using a pulley? 

• Draw the way pulleys can be installed on 
the given platform to help lift the weight 
shown below. 

Platform 


O 



• What happens to the string when pulleys 
are added to them? 

• What is a force? 

• What is velocity? 

• What is acceleration? 

• What is flow rate? 

• Which parameters affect flow rate? 

• Give a physical example of where regulat¬ 
ing flow rate is important and how it is 
regulated in practice? 

• Order the numbers from smallest to largest. 

• Order the numbers from largest to smallest. 

• Using the ruler measure the height of an 
object in inches. 

• What is the difference in height between 
the tallest and shortest objects? 

• Simplify the ratio of circumference to area. 

• How many digits does number n have? 

• Calculate the area of the circle. 

• What is an irrational number? 

• Calculate the mean. 

• Calculate the median. 

• Calculate the mode. 

• Solve a word problem with known average 
and missing data. 


Learning Standards 

PS 5.1b Observe and describe how the position 
or direction of motion of an object can be 
changed by pushing or pulling. 

PS 5.1c Observe how the force of gravity pulls 
objects toward the center of the Earth. 


PS 5.1a,b Patterns of motion, frame of 
reference and position, direction, and speed. 
PS 5.1b,c,d,e Newton’s Three laws of motion. 

S3.1 Tools in measurement. 

M2.1 Evaluating experimental results. 
Standard 6-3.2 Scientific notation. 


2.M.1 Use non-standard and standard units to 
measure both vertical and horizontal lengths. 
2.M.2 Use a ruler to measure standard units 
(including whole inches and whole feet). 

2.M.3 Compare and order objects according to 
the attribute of length. 

6.G.9 Understand the relationship between the 
circumference and the diameter of a circle. 

6.N.6 Understand the concept of ratio. 

6.R.1 Use physical objects, drawings, charts, 
tables, graphs, symbols, equations, or objects 
created using technology as representations. 

A.CN.7 Recognize and apply mathematical 
ideas to problem situations that develop outside 
of mathematics. 

A.S.l Categorize data as qualitative or quantitative. 
A.S.4 Compare and contrast the appropriateness 
of different measures of central tendency for a 
given data set. 


Table 1: Content questions used for the LEGO Mindstorms-based science and math 
activities and their associated New York City and State Science and Mathematics 
Learning Standards. 


by asking the same questions on the pre- and post-lesson assessment surveys. Questions 1-3 of the 
pre-lesson assessment survey were repeated in the post-lesson assessment survey for both science 
and math lessons. A binary-scale grading rubric (see Table 3) was used to evaluate the content 
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Assessment survey questions 

1. What gets you excited about science/math? 

2. If you were given the chance to create this lesson which method would you use: 

(a) Lecture; (b) Read textbook; (c) Watch movie; 

Pre-lesson (d) Conduct hands-on activity; (e) Research on the internet. 

Survey 

3. Do you think robotics can be helpful when used to collect data in science/math 

experiments? 

(a) Yes; (b) No; (c) Unsure. 


4. What did you like or dislike about the lesson? 

5. What did you like or dislike about the robotic device? 
Post-lesson 6. Rate this lesson using the following 

Survey (a) Strongly disliked; (b) Disliked; (c) Liked; (d) Strongly liked, 

(include 1-3 

above) 7. Do you think the use of robotics to collect data: 

(a) Made the lesson easier; (b) Made the lesson harder; 

(c) Made no difference in the lesson. 


Table 2: Pre- and post-lesson assessment survey evaluation questions 
for LEGO Mindstorms-based science and math activities. 


questions as either correct or incorrect. Similarly, for survey questions that required descriptive an¬ 
swers, student’s responses were analyzed and categorized as either Positive or Negative (Question 1) 
and Liked, Disliked, or No Response (Questions 4 and 5). Illustrative examples of students’ descriptive 
responses are provided in the Data Analysis and Observations section. 

The responses to survey questions were compiled and analyzed using bar charts and statisti¬ 
cal hypothesis tests. Bar charts summarize students’ performance before and after the activity. A 
dependent t-test [33] for paired samples was used to evaluate the difference in students’ average 
scores on the pre- and post-lesson assessment surveys content questions. Moreover, a McNemar’s 
test [33] on paired proportions was performed using the number of students that scored above 
and below the class average on the pre- and post-lesson assessment survey content questions. The 
results of both tests were used to examine the null hypothesis that no statistical difference between 
students’ scores on the pre- and post-lesson assessments existed. 


DATA ANALYSIS AND OBSERVATIONS 


The science activities outlined above were assessed in two 3rd grade elementary school sci¬ 
ence classes consisting of a total of 44 students, two 8th grade middle school technology classes 
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Content questions 


Correct responses 


Incorrect responses 


The Mechanical Advantage 

What is a pulley and what Student mentions keywords pertaining to a simple 
is the purpose of using a machine or a wheel in which a rope or string 
pulley? passes around. Student mentions how pulleys are 

used to achieve mechanical advantage or to lift 
heavy objects. Both parts of the questions must be 
answered by student to receive full credit. 

Draw the way pulleys can Student provides a diagram in which a rope or 

be installed on the given string goes over at least one upper pulley and under 

platform to help lift the at least one lower pulley in the diagram, 

weight shown below. 


What happens to the string Student discusses that the strings feel looser or 

when pulleys are added to strings are not as stiff or tight as before. 

them? 


What is a force? 


What is velocity? 


What is acceleration? 


What is flow rate? 


Acceleration Due to Gravity 

Student provides keywords pertaining to a pushing 
or pulling action, including making something 
move. 

Student provides keywords pertaining to the rate 
of change of position with respect to time. If the 
student uses the terms “fast” or “slow”, they must be 
used together, such as how fast or slow something 
is going. 

Student provides keywords pertaining to the rate 
of change of velocity with respect to time. If the 
student uses terms such as “speeding up” or 
“slowing down”, they must be used together, such 
as when something speeds up or slows down. 

Fluid Flow Rate 

Student provides keywords pertaining to the time 
required for a volume of liquid to enter (fill up) or 
exit (empty) a vessel. 


Which parameters affect Student provides keywords pertaining to the size 
flow rate? and shape of the orifice (nozzle), the amount of 

liquid in the container, and additional forces used to 
push or pull the liquid out, such as a pump. 


Give a physical example of 
where regulating flow rate 
is important and how it is 
regulated in practice. 


Student provides keywords pertaining to the 
movement of liquid, e.g., in a hose, a dam, a shower 
head, a river etc. The student must give examples 
of how flow rate is regulated in order to receive 
full credit, such as regulating flow rate is important 
when taking shower, the flow rate is regulated by 
the shower head and faucet. 


Student provides descriptions 
unrelated to those in the “Correct 
responses” column, including: 
“pulleys are used to pull things.” 


Student provides a drawing 
unrelated to the description in 
the “Correct responses” column 
that fails to yield mechanical 
advantage. 

Student gives an answer unrelated 
to the description in the “Correct 
responses” column, e.g., the strings 
become tighter and stiffen 


Student provides keywords or 
descriptions unrelated to those in 
the “Correct responses” column. 
Student provides keywords or 
descriptions unrelated to those in 
the “Correct responses” column. 


Student provides keywords or 
descriptions unrelated to those in 
the “Correct responses” column. 


Student provides keywords or 
descriptions unrelated to those in the 
“Correct responses” column, e.g., 
the rate at which something flows. 
Student lists parameters that have 
no relationship with the flow rate, 
e.g., material of the nozzle or the 
setup. 

Student provides keywords, 
descriptions, or examples unrelated 
to those in the “Correct responses” 
column. 


Table 3: Rubric for grading students’ responses to the pre- and post-lesson 


content questions. 


Contiunes 
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Content questions 

Correct responses 

Incorrect response 


Measurements & Accuracy 


Order the numbers from 

Student orders given numbers in increasing order. 

Student orders given numbers in 

smallest to largest. 


incorrect order, or leaves one or 
more numbers out. 

Order the numbers from 

Student orders given numbers in decreasing order. 

Student orders given numbers in 

largest to smallest. 


incorrect order, or leaves one or 
more numbers out. 

Using the ruler measure 

Student measures the object accurately and gives its 

Student measures the length of the 

the height of an object in 

length approximated to the nearest half-inch. 

object incorrectly with the reported 

inches. 


value being off by more than an 
inch. 

What is the difference in 

Student determines heights of given objects to the 

Student provides a difference in 

height between the tallest 

nearest hundreds and subtracts them to find their 

height of objects that deviates more 

and shortest objects? 

difference to the nearest hundreds. 

than a hundred from the correct 



answer. 


Pi - What is it? 


Simplify the ratio of 

Student correctly write the ratio in terms of radius r 

Student incorrectly writes the ratio 

circumference to area. 

and constant n and simplifies it to provide the final 

so that the simplified expression is 


expression in terms of radius r. 

incorrect, or incorrectly simplifies 
the expression, or writes numbers 
instead of an expression. 

How many digits does 

Student states that either the number is irrational or 

Student gives a finite number of 

number n have? 

that the digits after the decimal go on forever and 

digits, such as three digits as in 


never end, etc. 

“3.14”, or states that 7t has one 
hundred digits, etc. 

Calculate the area of the 

Student calculates the area correctly with an 

Student provides an answer that 

circle. 

answer given exactly in terms of n or numerically 

deviates greatly from the correct 


approximated. 

answer, especially by a factor, e.g., 
a multiple of 10. 

What is an irrational 

Student picks a number that cannot be turned into 

Student provides an answer 

number? 

a fraction or ratio; states that an irrational number 

unrelated to those in the “Correct 


is made up of a non-terminating/never-ending 

responses” column, refers to a 


decimal, e.g., 7t. 

number that is not irrational, e.g., 

-0.3. 


Means, Modes and Medians 


Calculate the mean. 

Student correctly calculates the mean of a given set 

Student gives an answer that differs 


of numbers using a calculator, or gives answer in 

from actual numerical mean and 


fractional format. 

performs calculation incorrectly. 

Calculate the median. 

Student correctly states or calculates the median of 

Student provides an incorrect value 


a given set of numbers. 

of the median of a given set of 
numbers. 

Calculate the mode 

Student provides the correct value of the mode of a 

Student provides an incorrect 


given set of numbers. 

value of the mode of a given set of 
numbers. 

Solve a word problem 

Student shows understanding of the problem. 

Student does not show an 

with known average and 

provides proper formula for finding the average, 

understanding of how to approach 

missing data. 

assigns a variable value to a unknown data in the set 
of values, and correctly solves for that unknown. 

the problem. 

Table 3: Continued 
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consisting of a total of 52 students, and two 10th grade high school science integration classes con¬ 
sisting of a total of 44 students. The math activities outlined above were assessed in three 2nd grade 
elementary school science and robotics classes consisting of a total of 41 students, one 6th and one 
7th grade middle school technology classes consisting of a total of 45 students, and three 9th grade 
high school algebra classes consisting of a total of 44 students. The activities were conducted in 
the classrooms where (1) students had limited prior knowledge of the subject matter facilitated by 
the LEGO Mindstorms setup or (2) teachers believed that the activity would serve as a beneficial 
supplementary instruction to the previously taught material. During the lesson, GK-12 Fellows and 
teachers provided timely instruction on the material needed for the students to understand the 
LEGO Mindstorms-based experiment and to be able to solve the supporting questions provided in 
the lab worksheet. For this study, each lesson with its corresponding pre- and post-assessment was 
conducted by employing two consecutive class sessions, wherein the assessments were conducted 
in two 20 minute periods before and after the 45 minute classroom activity. 

Students’ responses to content questions were graded following the grading rubric of Table 3 and 
illustrative examples of students’ graded works are available on our website for review. The bar chart 
representation of assessment results in Figures 7(a), (c), (e) and 8(a), (c), (e) show the percentage 
of students who responded correctly to pre- and post-activity content questions of science and 
math lessons, respectively. The assessment results given in Figures 7(b), (d), (f) and 8(b), (d), (f) 
show the percentage of students’ preferences to teaching methods before and after conducting the 
science and math activities, respectively. Figures 7(a), (c), (e) and 8(a), (c), (e) show that a strong 
transference of the lessons’ underlying concepts occurred during the LEGO Mindstorms-based 
science and math lessons, respectively. Note that the overall average increase in content question 
scores for the science and math based lessons was 47% and 25%, respectively. As evidenced by a 
dependent t-test [33] for paired samples and the McNemar’s test [33] on paired proportions (Table 
4) the improvement in the students’ knowledge on post-lesson assessment surveys was statistically 
significant. The tests were performed on the students’ average scores on content questions on the 
pre- and post-lesson assessment surveys. 

Figure 8(e) illustrates a large improvement in students’ understanding of the concept of mode 
in the Means, Modes, and Medians lesson with correct responses improving from 89% to 100%. 
This gain in students’ learning can be attributed to the ease of data collection facilitated through 
the use of LEGO Mindstorms ultrasonic sensor. This interaction with real data allowed students to 
couple mathematical concepts taught in the lesson with authentic use of collected numbers, as 
done by engineers and scientists [34], By extension this lesson was also helpful for students to 
learn the concept of frequency and multi modal data. The class average scores were high for other 
content questions during pre-evaluations for the lesson. This can be attributed to the fact that the 
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Content questions 


If you were given the chance to create this lesson, which method would you use? 



45% 

10% I 

sss^^H 


What is a pulley and what is Draw the way pulleys can be What happens to the string 
the purpose of using a installed on the platform to when pulleys are added to 

pulley? help lift the weight shown. them? 

Cl Pre-lesson assessment survey ■ Post-lesson assessment survey 



ci Pre-lesson assessment survey ■ Post-lesson assessment survey 


(a) 

Content questions 

90% 



What is What is 

force? velocity? acc 

s Pre-lesson assessment survey ■ Post-lesson assessment survey 


(b) 


If you were given the chance to create this lesson, which method would you use? 

100 % 



0 % 0 % 

Lecture Textbook Movie Hands-on Internet No 
reading activity response 

^ Pre-lesson assessment survey ■ Post-lesson assessment survey 


(c) 


Content questions 



What is flow rate? Which parameters affect Give a physical example of 
flow rate? where regulating flow rate is 

important? 


3 Pre-lesson assessment survey ■ Post-lesson assessment survey 


(d) 

If you were given the chance to create this lesson, which method would you use? 



Lecture Textbook Movie Hands-on Internet No 

reading activity response 


D Pre-lesson assessment survey ■ Post-lesson assessment survey 


(e) (f) 

Figure 7. Students’ response to pre- and post-lesson content questions assessment for 
(a) The Mechanical Advantage; (c) Acceleration due to Gravity; (e) Fluid Flow Rate activity 
and students’ response to pre-and post-lesson evaluation Question 2 for (b) The Mechanical 
Advantage; (d) Acceleration due to Gravity; and (f) Fluid Flow Rate activity. 
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Content questions 


If you were given the chance to create this lesson, which method would you use? 



Order objects: anal I Order objects: large Measurement Compare heights of 
to large to small estimation objects 


Lecture Textbook Movie Hands-on Internet No response 
reading activity 


3 fa-icuowastcttmcni survey ■ Poti-kwon oucssmau survey 


^Pie-lc**ofl*»C!<Mnca(aurvcy "P<ui-lc»ona«e»Mnetil survey 


(a) 

Content questions 


100% 

£* 



Simplify' a ratio How many digits Find the area of a What is an irrational 
docs x have? circle number? 


(b) 

If you were given the chance to create this lesson, which method would you use? 

100 % 

80% 


60% 

40% 



44.4% 

33-6% 



20% 

****.n4 

2.2% 2-2% 

H 

■ 

15.6% 

44%4.4% 

. W 


lecture 

Textbook 

reading 

Movie 

Hands-on 

activity 

Internet 

No response 


' Pre-kvionafcscttjncnl survey ■ POM-lewonMieumna survey 


- Piv-teumasKSMBicnt survey ■ Pust-taoun uxnmnl survey 


(c) 


(d) 



Central tendency; Central tendency; Central tendency: Average known Lecture Textbook Movie Hands-on Internet No response 

mean median mode with missing data reading activity 


' ftc-lwMin iMVtuiicnt survey 


■ Poil-lcsKaiufasinenl wey 


(e) 


(0 


Figure 8. Students’ response to pre- and post-lesson content questions assessment for 
(a) Measurements & Accuracy; (c) Pi-What is it?; (e) Means, Modes, and Medians activity and 
students’ response to pre-and post-lesson evaluation Question 2 for (b) Measurements & 
Accuracy; (d) Pi-What is it?; and (f) Means, Modes, and Medians activity. 


material was covered prior to the lesson and the activity served as a review for the Regents exam. 
The improvement on these concept questions was low but still significant as shown in Table 4. 
Similarly, Figure 8(a) illustrates a large improvement in students’ understanding of the concept of 
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Class content questions score statistics Significance Level 


Activity 

n 

Pre-lesson (%) 

Avg. St. dev. 

Post-lesson (%) 

Avg. St. dev. 

f-test 

McNemar' 

test 

The Mechanical Advantage 

42 

9 

18 

52 

28 

<0.001 

< 0.001 

Acceleration due to Gravity 

41 

33 

20 

91 

26 

<0.001 

< 0.025 

Fluid Flow Rate 

41 

11 

19 

51 

30 

<0.001 

< 0.001 

Measurements & Accuracy 

41 

62 

25 

81 

26 

<0.001 

<0.01 

Pi - What is it? 

45 

23 

21 

76 

30 

<0.001 

<0.01 

Means, Modes, and Medians 

44 

91 

17 

95 

10 

<0.05 

<0.1 


Table 4: Results of a dependent t-test for paired samples and McNemar’s test on 
paired proportions for the LEGO Mindstorms-based science and math activities. Here 
n is the number of student participants in the corresponding activity. 


descending order in the Measurements & Accuracy lesson with correct responses improving from 
41% to 80%. We believe that the combination of traditional instruction, students’ interaction with 
real data, and the presence of a student-friendly robot helped students to retain the knowledge of 
ordering numbers. 

Further analysis of Figures 7(b), (d), (f) and Figures 8(b), (d), (f) shows a consistent increase 
(pre- versus post-lesson) for hands-on activities as a preferred method for teaching across all les¬ 
sons. The post-lesson responses reported on all three school levels to the evaluation Question 1 had 
over 88% and 73% of the participating students displaying a positive attitude towards science and 
math, respectively. Moreover, the post-lesson responses reported on all three school levels to the 
evaluation Question 3 had over 66% and 80% of the participating students agreeing that robotic 
devices can be helpful for data collection in science and math, respectively. The “like” responses 
to Questions 4 and 5 regarding the robotic device and the lesson, accounted for more than 60% of 
all the responses among the three school levels for both science and math. The positive responses 
for Question 6 show that over 70% of the students rated their science lessons as at least “liked” 
and over 68% of the students rated their math lessons as at least “liked.” In response to Question 
7, regarding the ease of collecting data with the robot, more than 60% of the students reported 
that using robotics made both science and math lessons easier. Illustrative descriptive answers to 
Questions 1, 4, and 5 are provided in Table 5. 

The observations on student engagement from the GK-12 Fellows who facilitated the lessons 
are generally positive for both science and math lessons. Students were reported to be eager to 
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Activity 

Survey question 

Response category 

Descriptive answers 

The Mechanical 
Advantage 

What did you like or dislike 
about the lesson? 

Disliked 

“I didn’t understand some of 
the terms used.” 

Acceleration due to 
Gravity 

What did you like or dislike 
about the robotic device? 

Liked 

“I liked how it connected with 
Bluetooth.” 

Fluid Flow Rate 

What gets you excited about 
science? 

Positive 

“Realization of a problem that 

I always wondered about.” 

Measurements & 
Accuracy 

What did you like or dislike 
about the robotic device? 

Liked 

“It could draw and move.” 

Pi-What is it? 

What gets you excited about 
math? 

Positive 

“Using it in engineering.” 

Means, Modes, and 
Medians 

What did you like or dislike 
about the lesson? 

Disliked 

“I dislike robots.” 


Table 5: Illustrative answers to pre- and post-lesson assessment survey 
questions for LEGO Mindstorms-based science and math activities. 


participate in the lesson and actively encourage other classmates to join. They attentively listened 
to the lesson descriptions and completed the tasks assigned to them. As all activities were con¬ 
ducted using small groups, informed discussions often emerged where students tried to predict 
the behavior of the robot or debated conflicting observations on the experimental apparatuses. 
Students, without former experiences with LEGO Mindstorms, were able to quickly learn to op¬ 
erate the experimental devices and teach it to other students. GK-12 Fellows also reported that 
some students were uninterested in completing the pre- and post-evaluation surveys with similar 
questions. 


REFLECTIONS, SUSTAINABILITY, AND CONCLUSIONS 

The LEGO Mindstorms-based experiments were used to reinforce traditional classroom instruction 
of New York State mandated learning standards, typically during two consecutive 45 minute class 
sessions. The lessons can be adapted to support inquiry-based scientific exploration, depending on 
available time and students’ skill level. For example, in The Mechanical Advantage lesson students 
can be engaged in a discussion of the usage of pulleys in everyday life. Similarly, in the Pi-What is 
it? lesson, the program can be adjusted for circles with larger diameters, thus allowing students to 
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discover that n is a constant for all circles. The lessons’ descriptions available on our website provide 
supplemental enrichment content to promote inquiry. 

Even as numerous factors [35] are involved in determining whether a lesson is successfully 
communicated to students, the survey data reveals that students are intrigued by the use of LEGO 
Mindstorms in the science and math classroom setting. This suggests that students are inherently 
inquisitive in dealing with science and math topics and are interested in using their senses to learn 
about the subject in an explorative manner [36-37], The LEGO Mindstorms-based lessons facilitated 
students’ preferred methods of learning as it fostered their creativity while simultaneously estab¬ 
lishing boundaries and structure in accordance with the learning goals of the lesson. The evaluation 
surveys gauged students’ prior experiences with robotics and their beliefs about the usefulness of 
robotics as a tool for scientific inquiry and data acquisition. Moreover as students were more pre¬ 
disposed to lessons with a hands-on approach, the LEGO Mindstorms-based activities provided the 
additional effect of keeping students actively engaged throughout the lessons. 

Interestingly, students participating in the math based lessons preferred other modes of learn¬ 
ing equally and in some cases more than hands-on activities, as depicted in Figures 8(b), (d), (f), 
even though their attitudes towards the lesson and the robotics device were quite similar to that 
of the students in science classes. As depicted in Figures 7(b), (d), (f) and 8(b), (d), (f), the dif¬ 
ference in learning preferences for students involved in LEGO Mindstorms-based science versus 
math activities may stem from the students’ prior classroom experiences in the subject being 
taught. Science subjects rely on the interpretation of experimental data for explanations about 
the observed phenomena. Students readily expect to follow-up or precede a discussion of science 
topics with some experimentation to provide evidence that supports or disproves a claim; hence a 
hands-on activity facilitates this purpose. In math classes, such evidenced-based inquiries of the 
facts are often limited to analogous statements and portrayals. As such, students (and teachers) 
in math classes may be pre-conditioned to using other learning mechanisms in lieu of hands-on 
activities. 

From this study, two general principles for using LEGO Mindstorms, or similar robotic or experimental 
devices, in the classroom became apparent. First, a sufficient number of units must be made available 
to keep the ratio of number of students to the number of LEGO devices relatively low (e.g., 3 or lower). 
This ensures that all students remain engaged in the task at hand and can follow the structured and 
explorative components of the hands-on lesson, thus further reinforcing the goals of the lesson. Second, 
integration of technology such as LEGO Mindstorms in the classroom presupposes that the students 
possess a fundamental understanding of how the device functions so that they can fully utilize its 
capabilities. These two principles were observed in this study through (1) the availability of sufficient 
number of LEGO Mindstorms kits in the participating schools and (2) the collaboration between teachers 
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and GK-12 Fellows to ensure effective delivery of the lesson. Moreover, since the lessons outlined in this 
study were constrained to being delivered within in a single class period of approximately 45 minutes 
with an additional 40 minutes allocated for pre- and post-assessment, the programming and design 
component was eliminated from all lessons to ensure that all students had an equal starting point for 
participation in the lesson. Moreover, students were not permitted to change or modify the design or 
program to prevent the apparatus from being compromised and reporting inaccurate data. 

To ensure the sustainability of the use of LEGO Mindstorms in the classroom, project teachers have 
been provided with a foundation of robotics through participation in NYU-Poly conducted workshops. 
The activities introduced in this paper were conducted at schools that have already been participat¬ 
ing in FLL competitions, but did not previously use LEGO Mindstorms in a classroom setting to teach 
science and math. Therefore, we believe that the activities are sustainable in the participating schools 
without the continuing presence of a graduate Fellow. Moreover, we offer a step-by-step construc¬ 
tion guide and Mindstorms programs (available here) to assist teachers who possess experience in 
LEGO Mindstorms or have participated in FLL competitions in adopting these lessons. 

For teachers unfamiliar with LEGO Mindstorms, NYU-Poly offers periodic workshops. These 
workshops are open to teachers at all levels of expertise and are conducted by faculty and graduate 
students for introducing teachers to hands-on science and math activities using LEGO Mindstorms 
kit. As we conduct additional workshops and survey the workshop attendees, in a future paper, 
we will report on teachers' workshop experience and their successes or difficulties in integrating 
workshop activities in the classroom. 

To conclude, we recommend several directions for future work. First, the improvement in 
students’ responses to content questions provides an evidence of the effectiveness of LEGO 
Mindstorms-based experiments in teaching and learning of K-12 science and math subjects. A 
further study is necessary to establish whether students achieve long-term, sustainable learning 
gains through engagement with LEGO Mindstorms-based science and math lessons. Such a study 
requires engaging a cohort of students to conduct multiple LEGO Mindstorms-based science and 
math experiments throughout an academic year, which was not feasible in the present study due 
to logistical constraints. Second, as indicated in Figure 8(a) and (e), there were some instances 
where students’ performance did not change between the pre- and post- lesson assessment sur¬ 
veys, suggesting that consideration must be made for the adequate use of LEGO Mindstorms with 
respect to specific subject contexts. Finally, even though the pre- and post-lesson assessment survey 
results of our study indicate that the use of LEGO Mindstorms in science and math classrooms was 
generally received positively by the participating students, it would be informative to conduct a 
study where identical lessons are taught using conventional, didactic methods and then with the 
approach outlined in this paper. 
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